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IMPACT OF DIFFERENT PERFORMANCE ASSESSMENT CUT SCORES 



ON STUDENT PROMOTION 



Jia Wang, David Niemi, and Haiwen Wang 
CRESST/University of California, Los Angeles 



Abstract 

This cut score modeling study investigated whether changing the cut scores of 
language arts performance assessments (PAs) would differentially affect sub- 
groups of students. Based on a data set of 227,817 students, hierarchical logistic 
regression analysis revealed that no sub-groups of students were found to have a 
higher or lower possibility of passing the PAs based on their background variables 
when we switched between the two different cut scores, with one exception. 
Immigrant students were found to have a higher possibility of passing than the 
native students when the cut score was set at 3, and we found no differences 
between them with the cut score at 2. This result contributes to evidence on the 
relative fairness of the PAs. 



According to the general opinion of measurement specialists, student 
achievement on a test is more meaningful when considered as a continuous score 
rather than a small number of categorical scores (Linn, 2003). Green (1981) 
concluded that we should avoid fixed cut scores whenever possible. However, 
categorical scores cannot be avoided due to the application of some test results, and 
under certain situations pass-fail dichotomy is necessary in making decisions (Linn, 
2003). Since no absolute "true standard" or "best" method exists, it makes sense to 
inquire into the variability of the results according to different adopted methods 
(Linn, 2003). As Jaeger (1989) suggested, it is good practice to use several methods in 
one study and then compare the results with important factors before making 
decisions on a final cutoff score. 

This paper highlights findings on the impact of different cut scores on the 
likelihood of students passing a district-wide performance assessment in the Los 
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Angeles Unified School District (LAUSD) in Spring 2002. Specifically, we 
investigated whether setting the passing score as either a 2 or a 3 would have a 
different affect on different groups of students. For example, would certain 
subgroups of students be advantaged or disadvantaged by having 2 or 3 as the 
passing score? The results of this investigation address the fairness of the assessment 
and contribute to evidence on fairness obtained in other studies. 

Data 

There are 227,817 students in the final data set used for this study; 49% are 
male students and 51% are female students. All students in the data set were 
enrolled in Grades 2 through 9 at LAUSD during the school year 2001-2002. These 
students took language arts performance assignments (PAs). Students were 
included in the final data set for analyses if they were enrolled in Grades 2-9 and 
there were no missing values on all variables. The variables included in this study 
are: (a) grade level, (b) gender, (c) ethnicity, (d) meal program, (e) language 
classification, (f) Title 1 status, (g) immigrant status, (h) gifted student status, (i) 
home language survey results, 0 2001-2002 reading NCE score, (k) 2001-2002 math 
NCE score, (1) 2001-2002 language NCE score, and (m) 2001-2002 PA scores. 

Although close to 50% of the students in LAUSD are English Language 
Learners (ELLs), the data set used for this study does not have the same percentage 
of ELLs due to the district's testing eligibility requirements. Only ELL students at an 
advanced level of English language development are included in the PA testing, per 
district requirement. Therefore, in this report, the ELL students we refer to were 
actually the ELL students in the last stage of language development, right before 
being classified as Re-designated English Language Learners (RELLs). This final 
data set consequently includes only 16% ELL students. There were 35% RELLs and 
49% English proficient (EP) students (including those who were initially fluent in 
English and those who were English only students). 

Home language survey results confirm the language proficiency diversity in 
LAUSD. Based on the home language survey, 55% of the students reported speaking 
Spanish at home, 37% of the students reported speaking English, and 8% reported 
speaking languages other than Spanish and English at home. In terms of student 
racial and ethnic makeup, Hispanic students made up the largest sub-group of 
students (67%), followed by Blacks (16%), Whites (10%), Asians (4%), and students 
in none of the above ethnic groups (4%). 
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For the student distribution in the PAS, the majority of the students are in the 
"partially proficient" category (44% or 99,796 students). The next largest group 
consists of the 31% of the students who scored "proficient." Nine percent of the 
students scored "advanced" and 17% of the students scored "not proficient." 

For the student achievement scores in SAT-9 NCE, we found the mean reading, 
language, and math scores are 44, 49, and 49 points, respectively. The corresponding 
standard deviations for these three test scores are 18, 20, and 20 points, indicating a 
wide spread of student score distribution. Figure 1 shows students' mean SAT-9 
NCE scores in reading, math, and language by their performance category on the 
PAs. As shown, students who scored "advanced" on the PAs had the highest mean 
SAT-9 scores across all three subjects, and students in the category of "not 
proficient" had the lowest SAT-9 scores in reading, math, and language. There 
tended to be a 10-point increase in students' mean SAT-9 scores corresponding to 
each one-point increase in PA scores. The higher students scored in PAs, the higher 
they scored in SAT. This was one piece of descriptive evidence supporting the 
criterion validity of the PAs. 
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Figure 1. 2001-2002 students' mean scores by their PA categories. 



Descriptive Results 

Table 1 reported the percentages of students in the four categories of PA scores 
by their background variables. The percentages conveyed how students of a certain 
background group distributed across the four PA categories. These specific 
percentage numbers could be compared to the overall percentages on the top of the 
table to see how much deviation there was. Overall, 17% of the students were "not 
proficient," 44% were "partially proficient," 31% were "proficient," and 9% were 
"advanced." Results from Table 1 may be summarized as follows: 

Gender. Female students performed better in the PAs than male students. 
Proportionally, more female students fell in the categories of "proficient" and 
"advanced," and more male students had scores of "not proficient" and "partially 
proficient." 

Ethnicity. The majority of Asian, White, and Others students fell in the 
categories of "proficient" or "advanced," while the majority of Black and Hispanic 
students were in the categories of "partially proficient" and "proficient". This points 
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to the importance of examining where the cut score for proficiency should be since 
the decision seems to be ethnic-sensitive, at least descriptively. 

Free Meal Status. The free meal status variable was used as an indicator of 
students' family socio-economic status. Students who received free or reduced-fee 
lunch at school were over-represented in the lower categories of the PAs, being 
either "not proficient" or "partially proficient." Sixty-three percent of the students 
receiving free or reduced-fee lunch scored "not proficient" (18%) and "partially 
proficient" (45%), while the corresponding percentage for the students who pay for 
lunch was 54%. 

Title 1 Status. Title 1 status was another indicator of students' family socio- 
economic status. Students from disadvantaged families typically received Title 1 
services at school. Title 1 results were similar to results from the variable "free meal 
status," where students receiving Title 1 service were over-represented in the lower 
categories and under-represented in the higher categories of PAs. 

Language Proficiency. The distribution of ELL students across the four categories 
of PAs was very different from the distributions for REEL and EP students. Thirty- 
one percent of the ELL students, even though they were classified as at the 
"advanced" level of ELD, had a PA score of "not proficient"; 48% of ELL students 
are "partially proficient"; 18% are "proficient" and 3% are "advanced." Eor REEL 
students, the majority of students were "partially proficient" (44%) and "proficient" 
(34%). Only 13% of the students were in the "not proficient" category, and about 9% 
were "advanced" in the PAs. 

Immigrant Status. Immigrant status did not differentiate students in their PA 
performance as the other background variables did. They had similar performances 
on the PAs. 

Home Language Classification. Students whose home language was either 
English or Spanish had similar distribution patterns in their PA scores, with English- 
speaking students performing slightly better. English-speaking students had a 
higher proportion in the "proficient" and "advanced" categories, and lower 
proportion in "not proficient" and "partially proficient" categories. Students whose 
home languages were other than English and Spanish seemed to be in the higher 
achievers group. Among them, only 9% are "not proficient", 36% partially 
proficient, and 55% are either "proficient" or "advanced." 
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Grade Level. There was no uniform pattern in students' distribution across PA 
score categories for students of different grades. Students enrolled in Grades 2-5 and 
Grade 8 had more similar distributions than students in Grades 6, 7, and 9. Students 
in the first group had lower percentages in the categories of "not proficient" and 
higher percentages in "proficient" and "advanced," generally speaking. However, 
the reasons for this grade level effect were not clear. 

Gifted Status. Gifted students clearly did better than non-gifted students in the 
PAs. Gifted students had lower proportions of being "not proficient" and "partially 
proficient" and higher proportions in the other two proficient categories than non- 
gifted students. Among gifted students, only 4% were not proficient and 27% were 
partially proficient. While among non-gifted students, 19% were not proficient and 
46% were partially proficient. 
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Table 1 



The Distribution of Students' PA scores by their Background Variables, 2001-2002 (N=227,817) 





Nof 

Proficienf 


Partially 

Proficient 


Proficient 


Advanced 


Total 

Number of 
Sfudenfs 


Percenfage 
of Students 


Overall % 


17.0% 


43.8% 


30.6% 


8.6% 


227,817 




Gender 


Male 


20.6% 


45.7% 


27.3% 


6.4% 


111,126 


49% 


Female 


13.7% 


42.0% 


33.7% 


10.7% 


116,691 


51% 


Ethnicity 


All Other 


9.8% 


37.5% 


37.6% 


15.2% 


7,986 


4% 


Asian 


7.3% 


33.3% 


40.7% 


18.7% 


9,723 


4% 


Black, Not 


Hispanic 


21.3% 


44.7% 


27.0% 


7.0% 


35,857 


16% 


Hispanic 


18.1% 


45.6% 


29.4% 


7.0% 


152,078 


67% 


White, Not 


Hispanic 


10.0% 


37.0% 


37.5% 


15.5% 


22,173 


10% 


Free Meal Status 


Full pay 


14.1% 


40.3% 


33.5% 


12.2% 


58,903 


26% 


Free and 


Reduced 


18.1% 


45.0% 


29.5% 


7.3% 


168,914 


74% 


Title 1 Status 


Non-title 1 


13.2% 


38.9% 


34.5% 


13.3% 


59,076 


26% 


Title 1 


18.4% 


45.5% 


29.2% 


6.9% 


168,741 


74% 


Language 

Proficiency 


EP 


15.4% 


42.1% 


32.4% 


10.1% 


111,102 


49% 


REEL 


12.7% 


44.2% 


33.9% 


9.2% 


79,491 


35% 


ELL 


31.2% 


48.1% 


17.9% 


2.7% 


37,224 


16% 
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Table 1 (continued) 





Not 

Proficient 


Partially 

Proficient 


Proficient 


Advanced 


Total number 
of sfudenfs 


Percenfage of 
sfudenfs 


Immigrant or Native 


Non-immigrant 


16.8% 


43.8% 


30.8% 


8.6% 


195,724 


86% 


Immigrant 


18.7% 


43.7% 


29.1% 


8.6% 


32,093 


14% 


Home Language 


English 


16.8% 


42.8% 


31.0% 


9.5% 


85,368 


37% 


Spanish 


18.4% 


45.6% 


29.1% 


6.9% 


125,001 


55% 


Others 


9.0% 


36.2% 


38.6% 


16.2% 


17,448 


8% 


Grade Level 


Grade 2 


14.0% 


40.1% 


35.7% 


10.2% 


14,490 


6% 


Grade 3 


12.2% 


42.2% 


36.4% 


9.3% 


17,296 


8% 


Grade 4 


11.1% 


44.6% 


35.4% 


8.9% 


21,309 


9% 


Grade 5 


10.6% 


44.5% 


36.4% 


8.5% 


25,542 


11% 


Grade 6 


21.5% 


42.0% 


27.7% 


8.8% 


39,046 


17% 


Grade 7 


25.2% 


43.5% 


24.4% 


6.9% 


38,315 


17% 


Grade 8 


12.0% 


44.2% 


33.2% 


10.6% 


38,506 


17% 


Grade 9 


20.9% 


47.2% 


25.0% 


6.9% 


33,313 


15% 


Gifted Students 


Non-gif ted 


18.6% 


45.7% 


29.0% 


6.7% 


204,865 


90% 


Gifted 


3.5% 


27.1% 


44.3% 


25.1% 


22,952 


10% 



In summary, if the cut score for being proficient or not in the PAs is 3 
(proficient) instead of 2 (partially proficient), students of the following subgroups 
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were at the disadvantage of not passing the PAs when 3 was the cut score: males, 
Hispanics, students receiving free or reduced-pay lunch. Title 1 students, ELL 
students, students whose home language is Spanish, and non-gifted students. To 
test whether these observations were statistically significant when the other 
background variables were controlled for, we ran logistic regression analyses. 

Methods and Analysis 

Since each passing score (2 or 3) was a dichotomous dependent variable, we 
conducted a logistic regression analysis to take into consideration the complex and 
intertwining nature of the factors that impact learning. We used logistic regression 
to see whether there were differences in the significance of regression coefficients, in 
the adjusted odds ratio values, and in the partial possibilities with these two 
dependent variables. Various dummy variables were created to represent categorical 
variables on ethnicity, language proficiency, and home language classification. The 
estimation model we used takes the following form 

POSSIBILITY OL PASSING PERLORMANCE ASSIGNMENTS 

= a + [3 FEMALE + p ASIAN + p BLACK + p HISPANIC + [3 OTHERS 

+ [3 TITLE 1 + [3 IMMIGRANT + p FREE-LUNCH 

+ [3 SPANISH + p H_OTHER + p ELL + p RELL 

+ [3 GIFTED + p READING + [3 MATH + p LANGUAGE + e. 

Please note that £ in the preceding model indicates the base coefficient for the 
comparison group. The comparison group consisted of non-immigrant white male 
students who were not in Title 1, who paid for lunch, who were proficient in 
English, who were not gifted, whose home language was English, and who scored at 
the mean in SAT-9 reading, math, and language tests (NCE scale). Also, instead of 
using individual students' raw achievement scores, we used their transformed 
scores by deducting the mean from individuals' raw scores for the purpose of easier 
interpretation. 
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Figure 2. Comparison of partial possibility of passing the performance assignment 
assessment with cut scores at 2 and 3. 



Logistic Regression Results 

There were two dependent variables for the logistic regression analysis. One 
variable was determined by setting the passing score for the PAs at 2; the other by 
setting the passing score at 3. For both variables, students who passed are coded as 1 
and students who failed are coded as 0. 
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Logistic Regression Results - Passing at 2 

Except for the background variables on students' immigrant status and 
receiving free or reduced-fee lunch or not, all the other variables were significant 
predictors of whether a student had a passing PA score. As shown in Table 2, there 
was no statistical difference in students' likelihood of having a passing PA score 
between students who received a free or reduced-fee lunch and students who paid 
for lunch, and between immigrant students and native students. 

Female students were more likely to pass than male students. Students 
classified as "Asian and other" had a higher likelihood of passing, and Black and 
Hispanic students were more likely not to pass, compared to White students. 
Compared to students whose home language was English, students whose home 
language was Spanish and other than English or Spanish were more likely to pass 
the PAs. Compared to English proficient students, ELL students in the advanced 
stage of language development and REEL students were less likely to pass. 

Gifted students are more likely to pass than non-gifted students and Title 1 
students are less likely to pass than non-Title 1 students. In terms of previous 
achievement scores, students with a higher SAT-9 NCE reading, math, and language 
scores are more likely to pass the PAS than the other students. 
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Table 2 



Logistic Regression Results for Setting Performance Assignmenfs Passing Score af 2 or 3 



Passing af 2 or above Passing af 3 or above 



Partial Parfial 

Possibilify Possibilify 





Coefficienf 


Sfd. 

Error 


Adjusfed 
Odds Rafio 


Relafive fo 
fhe Base 


Coefficienf 


Sfd. 

Error 


Adjusfed 
Odds Rafio 


Relafive fo 
fhe Base 


Base 


















Possibilify 








82.0% 








26.7% 


Consfanf 


1.52’*' 


0.038 


4.6 




-i.or 


0.030 


0.0 




Female 


o.4r- 


0.012 


1.5 


5.3% 


0.43^^ 


0.010 


1.5 


9.2% 


Asian 


0.10’^ 


0.052 


1.1 


1.5% 


0.08^^ 


0.031 


1.1 


1.7% 


Black 


0.07’*' 


0.032 


1.1 


1.0% 


0.15^^ 


0.022 


1.2 


3.1% 


Hispanic 


0.09’^ 


0.033 


1.1 


1.3% 


0.06^^ 


0.023 


1.1 


1.2% 


Ofhers 


o.2r- 


0.047 


1.2 


2.9% 


0.23^^ 


0.030 


1.3 


4.8% 


Tiflel 


0.18’^ 


0.016 


1.2 


2.5% 


0.09^^ 


0.012 


1.1 


1.7% 


Immigranf 


0.03 


0.018 


1.0 


0.4% 


0.08^^ 


0.015 


1.1 


1.6% 


Lunch 


-0.01 


0.016 


1.0 


-0.2% 


-0.02’*' 


0.013 


1.0 


0.0% 


Spanish 


o.ir 


0.027 


1.1 


1.6% 


0.12’*' 


0.020 


1.1 


2.4% 


H_ofher 


0.17’*' 


0.044 


1.2 


2.4% 


O.IO’^ 


0.027 


1.1 


1.9% 


ELL 


-O.lb’^ 


0.026 


0.9 


-2.5% 


-O.IO’^ 


0.021 


0.9 


-1.9% 


RELL 


0.07’*' 


0.024 


1.1 


1.0% 


0.07‘*' 


0.016 


1.1 


1.4% 


Giffed 


0 . 19 * 


0.039 


1.2 


2.7% 


0.07’*' 


0.018 


1.1 


1.3% 


Reading 


0 . 03 * 


0.001 


1.0 




0.03^^ 


0.001 


1.0 
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Table 2 (continued) 



Passing at 2 or above Passing at 3 or above 



Partial Partial 

Possibility Possibility 





Coefficienf 


Sfd. 

Error 


Adjusfed 
Odds Rafio 


Relafive fo 
fhe Base 


Coefficienf 


Sfd. 

Error 


Adjusfed 
Odds Rafio 


Relafive fo 
fhe Base 


Mafh 


0.02’*' 


0.001 


1.0 




0.02’*' 


0.000 


1.0 




Language 


0.02’*' 


0.001 


1.0 




0.02’*' 


0.000 


1.0 





Note 1. The base possibility is calculated for a non-immigrant male White student who is not in Title 
1, who pays for lunch, who is proficienf in English, who is nof giffed, whose home language is 
English, and who scored fhe mean values in reading, mafh, and language. 2. Variable coefficienfs 
denofed wifh indicafe fhe variables are significanf af 0.05 significance level. 



Besides reporting regression coefficients, standard errors, and adjusted odds 
ratios. Table 3 also presented the partial possibilities associated with each of the 
variables and the base possibility for the comparison group. The base comparison 
group for this logistic regression analyses consisted of students who were (a) non- 
immigrant, (b) male, (c) white, (d) not enrolled in Title 1, (e) not receiving free or 
reduced-fee lunch, (f) English proficient, and (g) had mean NCE scores. As shown in 
Table 3, the possibility for students in the base comparison group to pass the PAs at 
score of 2 was 82%. This corresponds to the fact that in the actual data, 83% of the 
students score 2 or higher in the PAs. 

The specific numbers under the Column "Partial Possibility Relative to the 
Base" (corresponding to all predictors except regression constant) indicated the 
incremental possibility associated with each predictor. Eor example, holding 
everything else constant and comparing to the base comparison group of students, 
female students had a 5.3% higher possibility of passing the PAs at Score 2 than 
male students; gifted students had a 2.7% higher possibility of passing at Score 2 
than non-gifted students; and the ELL students in the advanced stage of English 
language development had a 2.5% lower possibility of passing at Score 2 than EP 
students. 
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Logistic Regression Results - Passing at 3 

Similar results were found when the cut score is set at 3, as reported in the last 
four columns of Table 2, instead of 2 for passing the PAs with one exception. When 
the cut score is 3, immigrant students were statistically more likely to pass the PAs 
relative to native students. In other words, native students would be at a 
disadvantage if the cut score is 3. All the other predictors stayed statistically 
significant for both outcome variables. In turn, two additional observations are in 
order. One, the standard errors for all predictors went down when the cut score is 3. 
This indicated that the effects of students' background, family, and school variables 
on their possibility of passing the PAs became more reliable. The other possibility is 
that the regression coefficients change their sizes up and down without any uniform 
pattern. This implies that the subgroup differences in their chances of passing the 
PAs are similar across these two outcome variables overall. 

When the cut score for passing the PAs is 3, the possibility for students in the 
base comparison group to pass is 26.7%. This is much lower than the base possibility 
of passing when the cut score is 2, as expected. Descriptively, only 39.2% of the 
students in this data set have PA scores of 3 and 4. 

Relative to the partial possibility values when the outcome variable has a 2 as 
the passing score, the magnitude of the partial possibility values increased for seven 
variables and decreased for the remaining six variables, excluding the three NCE 
achievement variables. For example, holding everything else constant, female 
students have a 9.2% (compared to 5.3% with 2 as the passing score) higher 
possibility of passing the PAs than male students; and ELL students in the advanced 
stage of English language development have a 1.9% (2.5% with passing score at 2) 
lower possibility of passing than EP students. 

To demonstrate how students' SAT-9 achievement scores affected students' 
possibilities of passing the PAs, Figure 1 graphs the possibilities of passing for the 
base comparison group of students. The three lines on the top of Figure 1 have the 
possibility lines when the passing score for the PAs is 2. When the passing score is 2 
instead of the higher score of 3, more students pass. The three lines on the bottom 
show the possibilities of passing the PAs relative to students' SAT-9 scores when the 
cut score is 3. As shown in the figure, reading scores have the sharpest slopes, 
indicating that the possibility of passing PA scores is most responsive to students' 
reading scores, compared to language and math scores. For a comparison group 
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student, the chance of passing the PAs with the cut score of 3 is 10% with a SAT-9 
reading score of 1, and the possibility increases to 63% when the student's SAT-9 
reading score is 99. Similar results are found for both language and math scores, 
regardless of whether the cut score is 2 or 3. 

Logistic Regression Results by Grade Level 

Since there is no uniform distribution pattern for students in different grades, 
and considering that grade level usually makes a difference in students' 
achievement performance, we also ran the logistic regression by grade level. We 
found by-grade variation in regression coefficients for all variables in terms of either 
statistical significance and/or coefficient values for both dependent variables. For 
example when the dependent variable is passing at 2, we noticed that there are four 
variables (Female, Black, Title 1, and Spanish-as-home-language) that have 
significant coefficients across all grade levels, and the coefficient values vary across 
grades. The female coefficients range from 0.42 (Grade 3) to 0.62 (Grade 4); the 
coefficients for Black range from -0.88 (Grade 8) to -0.28 (Grade 5); the coefficients 
for Title 1 range from -1.68 (Grade 2) to -0.15 (Grade 9); and the Spanish coefficients 
range from 0.19 to 0.54. For all the other variables, their by-grade regression 
coefficients not only vary in statistical significance, but also in coefficient values. A 
similar pattern is found for the dependent variable with 3 as the passing score. 

The following list provide only the grades that we found statistical significant 
for the sub-group of students in their possibility of passing the PAs or not: 

1. Asian students in Grades 2, 5 and 6. When the cut score is set at 3, Asian 
students in Grades 2, 5, and 6 are statistically more likely to pass the PAs 
than White students. 

2. Black students in Grades 3, 6 and 7. When the cut score is set at 3, Black 
students in Grades 3 and 6 are statistically more likely to pass the PAs 
than White students. When the cut score is set at 2, Black students in 
Grade 7 are statistically less likely to pass the PAs than White students. 

3. Hispanic students in Grades 3, 4, 7 and 8. When the cut score is set at 3, 
Hispanic students in Grade 3 are statistically more likely to pass the 
PAs. When the cut score is set at 2, Hispanic students in Grade 4 are 
statistically more likely to pass than White students and Hispanic 
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students in Grades 7 and 8 are statistically less likely to pass the PAs 
than White students. 

4. Students of other ethnic groups in Grades 2-3 and 6-9. When the cut score is 
set at 3, students of other ethnicity in Grades 2, 3, 6, 7, and 8 are 
statistically more likely to pass the PAs than White students. When the 
cut score is set at 2, students of other ethnicity in Grade 9 are more likely 
to pass. 

5. Title 1 students in Grades 4 and 5. When the cut score is 3, Title 1 students 
in Grades 4 and 5 are statistically more likely to pass. 

6. Immigrant students in Grades 6 and 8. When the cut score is 3, immigrant 
students in Grade 6 are statistically more likely to pass the PAs. 

7. Students who speak Spanish at home in Grade 8. When the cut score is 3, 
grade 8 students who speak Spanish at home are statistically more likely 
to pass the PAs than their peers. 

8. Students speaking languages other than English and Spanish at home in Grades 

2, 3, 5 and 8. When the cut score is set at 3, and 5*-grade 

students whose home language is other than English and Spanish are 
statistically more likely to pass the PAs than students whose home 
language is English. When the cut score is 2, English-speaking 8*-grade 
students are statistically more likely to pass when the cut score is 2. 

9. ELL students in Grades 5 and 8. When the cut score is at 3, ELL students in 
Grade 5 are statistically more likely to pass the PAs than EP students 
while Grade 8 ELL students are statistically less likely to pass. 

10. REEL students in Grades 2, 8 and 9. When the cut score is at 3, REEL 
students in Grade 2 have a statistically higher likelihood of passing than 
EP students. When the cut score is at 2, REEL have a statistically lower 
possibility of passing the PAs. 

11. Gifted students in Grades 2, 3, 5, 6, and 7. When the cut score is at 3, Grade 
3 gifted students, have a statistically higher possibility of passing the 
PAs than non-gifted students. When the cut score is at 3, Grades 6 and 7 
gifted students have a statistically higher likelihood of passing, and 
gifted students in Grade 2 are statistically less likely to pass the PAs. 



16 




Summary and Conclusions 



In summary, students in the following sub-groups have a higher possibility of 
passing the PAs: (a) female students are more likely to pass than male students; (b) 
White students have a lower possibility of passing than the other students; (c) Title 1 
students are more likely to pass than non-Title 1 students; (d) students whose home 
language is English are less likely to pass the PAs when compared to students 
whose home language is not English; (e) students who are English proficient are 
more likely to pass than ELL students in the advanced stage of language 
development and are less likely to pass than REEL students; (f) gifted students are 
more likely to pass than non-gifted students; and (g) students with higher SAT-9 
scores are more likely to pass than students with lower scores. All these 
interpretations are valid under the condition that all other variables in the model 
specification are held constant. Please also note that even though we found 
statistically significant differences in the above groups of students, a 1% or 2% 
difference in the possibility calculation is not large enough to be substantially 
important. 

As mentioned above, these student background variables, family variables, and 
student characteristic variables all have statistically significant effects on the 
possibility of passing the PAs in 2002, regardless of whether the cut score is 2 or 3. 
There are, however, two exceptions. One is that students' immigrant status was 
found to be insignificant when the cut score is 2, and it was significant when the cut 
score is 3. The statistical significance changed with the change of the cut score. 
Immigrant students are more likely to pass the PAs when the cut score is 3 than 
students born in the United States. 

The other exception is that participation in a school lunch program is the only 
variable that has no statistically significant effect on students' passing the PAs at 
either cut score. This is somewhat surprising since as a typical proxy for student 
socio-economic status (SES), school lunch program indicator is usually a significant 
predictor of students' school performance, despite the fact that it is not the best 
measure of SES. We explored the exclusion of students' Title 1 Status in the model, 
and found that the lunch program variable remained insignificant. One possible 
explanation for this finding could be that the PAs is not sensitive to students' family 
social-economic status, as represented by whether students received free or reduced- 
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fee lunch. These findings are based on the model without taking into consideration 
students' grade level. 

The by-grade analysis is more detailed, but provided no clear and consistent 
pattern of how these independent variables affect students' possibility of passing the 
PA differently when the cut score is changed between Score 2 and Score 3. For 
example, we found that across different grades, the same variable would be 
statistically significance for some grades and not significant for others, and it could 
be positively related to the outcome variables for some grades and negatively 
related to the outcome variables for other grades, besides having different sizes of 
the coefficients. In other words, certain subgroups of students do become the 
disadvantaged or advantaged groups when we switched between the two cut 
scores. 

The findings summarized above indicate that no matter where we set the cut 
score, most sub-groups of students would have the similar possibilities of passing 
the PAs statistically at either cut Score 2 or 3. Practically, however, caution should be 
exercised in making the decision considering that the decision will involve about 
100,000 students who Scored 2 in 2002 in our sample and that the PAs would also 
contribute to students' chances of being retained or promoted to the next grade. 
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